User experience oriented path selection

ABSTRACT

Methods and systems for path selection involving remote access protocols and/or user behavior are described herein. A request, from a first computing device, for content hosted on a second computing device may be received. Based on network state metrics, remote access protocol metrics, and/or user experience metrics, a path of a plurality of paths between the first computing device and the second computing device may be selected. The path need not be the most direct path between the first computing device and the second computing device, and may comprise remote access to a computing device on an intermediary server. Based on user behavior analysis performed with respect to user input data, a path may be re-selected, and/or the network state metrics, remote access protocol metrics, and/or user experience metrics may be weighted.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to International Application No.PCT/CN2019/077609, filed Mar. 11, 2019, and entitled “User ExperienceOriented Path Selection,” which is hereby incorporated by reference asto its entirety.

FIELD

Aspects described herein generally relate to computer networking, suchas network path selection, network state metrics, remote access tocomputing devices, and user input monitoring.

BACKGROUND

Typical protocols for delivering content over a network (e.g., HypertextTransfer Protocol (HTTP)) are often undesirably slow. Such slowness isparticularly the case over long distances and where requested content(e.g., a web page) requires a large number of requests and/or where thecontent itself is large. For example, a computer user in China mayaccess a web application hosted on a server in the United States, suchthat retrieval of any given element from the server may take 300milliseconds. In such an example, because the web application maycomprise a large number of elements (e.g., Hypertext Markup Language(HTML) code, Cascading Style Sheets (CSS), images, video content, andthe like), each element may entail 300 milliseconds of delay, meaningthat a user may experience significant delay when accessing the webapplication.

Remote access systems (e.g., servers which allow users to accessapplications hosted on the server from a remote computing device) havebecome increasingly popular, particularly as network speeds andcapabilities have improved. For example, a user on a relatively weakcomputing device (e.g., a thin client) might remotely access anapplication executing on a relatively powerful server and thereby usethe application as if the relatively weak computing device was executingthe application. As another example, a user might use a remote webbrowser to access a web page, such that the user's connection to the webpage involves at least two hops: a first hop via a remote accessprotocol to the server executing the web browser, and a second hop viathe web browser and via a different protocol from the server to a secondserver hosting the web page.

SUMMARY

The following presents a simplified summary of various aspects describedherein. This summary is not an extensive overview, and is not intendedto identify required or critical elements or to delineate the scope ofthe claims. The following summary merely presents some concepts in asimplified form as an introductory prelude to the more detaileddescription provided below.

As described above, remote access scenarios are often reliant on thespeed and reliability of network connections between computing devices(e.g., the relatively weak computing device and the relatively powerfulserver). But establishing and using such remote access applications canbe time-consuming and difficult, particularly for users unfamiliar withthe technology. Moreover, changes in network conditions and user needsmay mean that the right method of accessing content (e.g., directly, viaa remote web browser hosted on an intermediary computing device, or viaa second remote web browser hosted on a different intermediary computingdevice) may change.

To overcome these limitations, and to overcome other limitations thatwill be apparent upon reading and understanding the presentspecification, aspects described herein are directed towards pathoptimization, using remote access protocols, in networks. A request, bya first computing device, for content on a second computing device maybe received. The request may be for, e.g., a web page hosted on thesecond computing device. Network state metrics, such as latency,bandwidth, and the like, may be determined for one or more paths betweenthe first computing device and the second computing device. Such pathsmay comprise, e.g., routers, switches, intermediary servers, and thelike. Remote access protocol metrics, such as latency information forremote access protocols between a remote access client and a remoteaccess server, may be determined. Based on the network state metrics andthe remote access protocol metrics, user experience metrics may bedetermined. Such user experience metrics may comprise determining, froma plurality of paths in the network between the first computing deviceand the second computing device, one or more paths which may provide anoptimized user experience. Based on the network state metrics, theremote access protocol metrics, and the user experience metrics, a pathmay be selected, and the first computing device may access the contentvia the path. The path may comprise using a remote access protocol toaccess an application on an intermediary server. For example, the pathmay comprise the first computing device using a remote access protocolto access a web browser application executing on an intermediarycomputing device, wherein the web browser application is used by thefirst computing device to access the content on the second computingdevice. User input data from the first computing device may be receivedand, based on a user behavior analysis performed based on such userinput data, it may be determined that the user experience via the pathis poor. Based on such a determination, one or more reasons for the pooruser experience may be determined, and a new path may be selected.

These and additional aspects will be appreciated with the benefit of thedisclosures discussed in further detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of aspects described herein and theadvantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers indicate like features, and wherein:

FIG. 1 depicts an illustrative computer system architecture that may beused in accordance with one or more illustrative aspects describedherein.

FIG. 2 depicts an illustrative remote-access system architecture thatmay be used in accordance with one or more illustrative aspectsdescribed herein.

FIG. 3 depicts an illustrative virtualized (hypervisor) systemarchitecture that may be used in accordance with one or moreillustrative aspects described herein.

FIG. 4 depicts an illustrative remote access environment comprising afirst computing device, a second computing device, and an intermediarycomputing device.

FIG. 5 depicts three illustrative paths between a first computing deviceand a second computing device.

FIG. 6 is a flow chart which may be performed with respect to a firstcomputing device and a second computing device.

FIG. 7 is a process flow chart for a first computing device and a secondcomputing device.

FIG. 8 is a high level flow chart for path selection.

FIG. 9 depicts a process for user behavior analysis based on the userinput data.

DETAILED DESCRIPTION

In the following description of the various embodiments, reference ismade to the accompanying drawings identified above and which form a parthereof, and in which is shown by way of illustration various embodimentsin which aspects described herein may be practiced. It is to beunderstood that other embodiments may be utilized and structural andfunctional modifications may be made without departing from the scopedescribed herein. Various aspects are capable of other embodiments andof being practiced or being carried out in various different ways.

As a general introduction to the subject matter described in more detailbelow, aspects described herein are directed towards improving access tocontent on a network using remote access protocols. Computing devicesmay be configured to deliver content, such as a web page. Computingdevices may additionally and/or alternatively be configured to provideremote access to applications, such as web browsers, and suchapplications may request and retrieve content from other computingdevices. In this manner, a computing device may use an application on anintermediate computing device, via a remote access protocol, to accesscontent on a second computing device.

A user that uses a computing device to access content on a secondcomputing device over a network may have their experience adverselyimpacted by poor network conditions. For example, a user in China thatattempts to access a web page hosted in the United States may find thatthe latency associated with the distance between the two countries maycause an undesirable amount delay in loading the web page. In somecircumstances, the aforementioned delay may be improved by using anintermediary connection and/or a remote access protocol (e.g., remotelyaccessing a web browser on an intermediary computing device in Germany,then using that web browser to load the aforementioned web page), butdetermining that latency may be improved using such a method andestablishing such a connection may be prohibitively difficult and/ortime consuming. Moreover, even if one method (e.g., using the remote webbrowser to access a web page) does improve the user's experience,changing network conditions (e.g., less traffic on a network) may meanthat a different method (e.g., directly accessing the web page) mayresult in a better user experience at a later time.

Aspects described herein relate to path optimization in networks andusing remote access protocols. By evaluating network state metrics,remote access protocol metrics, and/or user experience metrics, a path,of a plurality of paths, may be selected which improves access tocontent. Based on user input data retrieved with respect to the content,path selection may be further improved. Additionally, aspects describedherein may learn from user experience (e.g., whether or not a user has apoor experience with a given path) Thus, in the above example, a usermay have an improved experience with access to content and need notmanually determine and select one of a plurality of possible paths tothe content. For example, a user may select an icon to open a web pageand, for user experience reasons, access such a web page via a webbrowser executing on an intermediary computing device withoutspecifically deciding to do so.

It is to be understood that the phraseology and terminology used hereinare for the purpose of description and should not be regarded aslimiting. Rather, the phrases and terms used herein are to be giventheir broadest interpretation and meaning. The use of “including” and“comprising” and variations thereof is meant to encompass the itemslisted thereafter and equivalents thereof as well as additional itemsand equivalents thereof. The use of the terms “connected,” “coupled,”and similar terms, is meant to include both direct and indirectmounting, connecting, coupling, positioning and engaging.

Computing Architecture

Computer software, hardware, and networks may be utilized in a varietyof different system environments, including standalone, networked,remote-access (also known as remote desktop), virtualized, and/orcloud-based environments, among others. FIG. 1 illustrates one exampleof a system architecture and data processing device that may be used toimplement one or more illustrative aspects described herein in astandalone and/or networked environment. Various network nodes 103, 105,107, and 109 may be interconnected via a wide area network (WAN) 101,such as the Internet. Other networks may also or alternatively be used,including private intranets, corporate networks, local area networks(LAN), metropolitan area networks (MAN), wireless networks, personalnetworks (PAN), and the like. Network 101 is for illustration purposesand may be replaced with fewer or additional computer networks. A localarea network 133 may have one or more of any known LAN topology and mayuse one or more of a variety of different protocols, such as Ethernet.Devices 103, 105, 107, and 109 and other devices (not shown) may beconnected to one or more of the networks via twisted pair wires, coaxialcable, fiber optics, radio waves, or other communication media.

The term “network” as used herein and depicted in the drawings refersnot only to systems in which remote storage devices are coupled togethervia one or more communications paths, but also to stand-alone devicesthat may be coupled, from time to time, to such systems that havestorage capability. Consequently, the term “network” includes not only a“physical network” but also a “content network,” which is comprised ofthe data—attributable to a single entity—which resides across allphysical networks.

The components may include data server 103, web server 105, and clientcomputers 107, 109. Data server 103 provides overall access, control andadministration of databases and control software for performing one ormore illustrative aspects describe herein. Data server 103 may beconnected to web server 105 through which users interact with and obtaindata as requested. Alternatively, data server 103 may act as a webserver itself and be directly connected to the Internet. Data server 103may be connected to web server 105 through the local area network 133,the wide area network 101 (e.g., the Internet), via direct or indirectconnection, or via some other network. Users may interact with the dataserver 103 using remote computers 107, 109, e.g., using a web browser toconnect to the data server 103 via one or more externally exposed websites hosted by web server 105. Client computers 107, 109 may be used inconcert with data server 103 to access data stored therein, or may beused for other purposes. For example, from client device 107 a user mayaccess web server 105 using an Internet browser, as is known in the art,or by executing a software application that communicates with web server105 and/or data server 103 over a computer network (such as theInternet).

Servers and applications may be combined on the same physical machines,and retain separate virtual or logical addresses, or may reside onseparate physical machines. FIG. 1 illustrates just one example of anetwork architecture that may be used, and those of skill in the artwill appreciate that the specific network architecture and dataprocessing devices used may vary, and are secondary to the functionalitythat they provide, as further described herein. For example, servicesprovided by web server 105 and data server 103 may be combined on asingle server.

Each component 103, 105, 107, 109 may be any type of known computer,server, or data processing device. Data server 103, e.g., may include aprocessor 111 controlling overall operation of the data server 103. Dataserver 103 may further include random access memory (RAM) 113, read onlymemory (ROM) 115, network interface 117, input/output interfaces 119(e.g., keyboard, mouse, display, printer, etc.), and memory 121.Input/output (I/O) 119 may include a variety of interface units anddrives for reading, writing, displaying, and/or printing data or files.Memory 121 may further store operating system software 123 forcontrolling overall operation of the data processing device 103, controllogic 125 for instructing data server 103 to perform aspects describedherein, and other application software 127 providing secondary, support,and/or other functionality which may or might not be used in conjunctionwith aspects described herein. The control logic 125 may also bereferred to herein as the data server software 125. Functionality of thedata server software 125 may refer to operations or decisions madeautomatically based on rules coded into the control logic 125, mademanually by a user providing input into the system, and/or a combinationof automatic processing based on user input (e.g., queries, dataupdates, etc.).

Memory 121 may also store data used in performance of one or moreaspects described herein, including a first database 129 and a seconddatabase 131. In some embodiments, the first database 129 may includethe second database 131 (e.g., as a separate table, report, etc.). Thatis, the information can be stored in a single database, or separatedinto different logical, virtual, or physical databases, depending onsystem design. Devices 105, 107, and 109 may have similar or differentarchitecture as described with respect to device 103. Those of skill inthe art will appreciate that the functionality of data processing device103 (or device 105, 107, or 109) as described herein may be spreadacross multiple data processing devices, for example, to distributeprocessing load across multiple computers, to segregate transactionsbased on geographic location, user access level, quality of service(QoS), etc.

One or more aspects may be embodied in computer-usable or readable dataand/or computer-executable instructions, such as in one or more programmodules, executed by one or more computers or other devices as describedherein. Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types when executed by a processor ina computer or other device. The modules may be written in a source codeprogramming language that is subsequently compiled for execution, or maybe written in a scripting language such as (but not limited to)HyperText Markup Language (HTML) or Extensible Markup Language (XML).The computer executable instructions may be stored on a computerreadable medium such as a nonvolatile storage device. Any suitablecomputer readable storage media may be utilized, including hard disks,CD-ROMs, optical storage devices, magnetic storage devices, and/or anycombination thereof. In addition, various transmission (non-storage)media representing data or events as described herein may be transferredbetween a source and a destination in the form of electromagnetic wavestraveling through signal-conducting media such as metal wires, opticalfibers, and/or wireless transmission media (e.g., air and/or space).Various aspects described herein may be embodied as a method, a dataprocessing system, or a computer program product. Therefore, variousfunctionalities may be embodied in whole or in part in software,firmware, and/or hardware or hardware equivalents such as integratedcircuits, field programmable gate arrays (FPGA), and the like.Particular data structures may be used to more effectively implement oneor more aspects described herein, and such data structures arecontemplated within the scope of computer executable instructions andcomputer-usable data described herein.

With further reference to FIG. 2, one or more aspects described hereinmay be implemented in a remote-access environment. FIG. 2 depicts anexample system architecture including a computing device 201 in anillustrative computing environment 200 that may be used according to oneor more illustrative aspects described herein. Computing device 201 maybe used as a server 206 a in a single-server or multi-server desktopvirtualization system (e.g., a remote access or cloud system) and can beconfigured to provide virtual machines for client access devices. Thecomputing device 201 may have a processor 203 for controlling overalloperation of the device 201 and its associated components, including RAM205, ROM 207, Input/Output (I/O) module 209, and memory 215.

I/O module 209 may include a mouse, keypad, touch screen, scanner,optical reader, and/or stylus (or other input device(s)) through which auser of computing device 201 may provide input, and may also include oneor more of a speaker for providing audio output and one or more of avideo display device for providing textual, audiovisual, and/orgraphical output. Software may be stored within memory 215 and/or otherstorage to provide instructions to processor 203 for configuringcomputing device 201 into a special purpose computing device in order toperform various functions as described herein. For example, memory 215may store software used by the computing device 201, such as anoperating system 217, application programs 219, and an associateddatabase 221.

Computing device 201 may operate in a networked environment supportingconnections to one or more remote computers, such as terminals 240 (alsoreferred to as client devices and/or client machines). The terminals 240may be personal computers, mobile devices, laptop computers, tablets, orservers that include many or all of the elements described above withrespect to the computing device 103 or 201. The network connectionsdepicted in FIG. 2 include a local area network (LAN) 225 and a widearea network (WAN) 229, but may also include other networks. When usedin a LAN networking environment, computing device 201 may be connectedto the LAN 225 through a network interface or adapter 223. When used ina WAN networking environment, computing device 201 may include a modemor other wide area network interface 227 for establishing communicationsover the WAN 229, such as computer network 230 (e.g., the Internet). Itwill be appreciated that the network connections shown are illustrativeand other means of establishing a communications link between thecomputers may be used. Computing device 201 and/or terminals 240 mayalso be mobile terminals (e.g., mobile phones, smartphones, personaldigital assistants (PDAs), notebooks, etc.) including various othercomponents, such as a battery, speaker, and antennas (not shown).

Aspects described herein may also be operational with numerous othergeneral purpose or special purpose computing system environments orconfigurations. Examples of other computing systems, environments,and/or configurations that may be suitable for use with aspectsdescribed herein include, but are not limited to, personal computers,server computers, hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, set top boxes, programmable consumerelectronics, network personal computers (PCs), minicomputers, mainframecomputers, distributed computing environments that include any of theabove systems or devices, and the like.

As shown in FIG. 2, one or more client devices 240 may be incommunication with one or more servers 206 a-206 n (generally referredto herein as “server(s) 206”). In one embodiment, the computingenvironment 200 may include a network appliance installed between theserver(s) 206 and client machine(s) 240. The network appliance maymanage client/server connections, and in some cases can load balanceclient connections amongst a plurality of backend servers 206.

The client machine(s) 240 may in some embodiments be referred to as asingle client machine 240 or a single group of client machines 240,while server(s) 206 may be referred to as a single server 206 or asingle group of servers 206. In one embodiment a single client machine240 communicates with more than one server 206, while in anotherembodiment a single server 206 communicates with more than one clientmachine 240. In yet another embodiment, a single client machine 240communicates with a single server 206.

A client machine 240 can, in some embodiments, be referenced by any oneof the following non-exhaustive terms: client machine(s); client(s);client computer(s); client device(s); client computing device(s); localmachine; remote machine; client node(s); endpoint(s); or endpointnode(s). The server 206, in some embodiments, may be referenced by anyone of the following non-exhaustive terms: server(s), local machine;remote machine; server farm(s), or host computing device(s).

In one embodiment, the client machine 240 may be a virtual machine. Thevirtual machine may be any virtual machine, while in some embodimentsthe virtual machine may be any virtual machine managed by a Type 1 orType 2 hypervisor, for example, a hypervisor developed by CitrixSystems, IBM, VMware, or any other hypervisor. In some aspects, thevirtual machine may be managed by a hypervisor, while in other aspectsthe virtual machine may be managed by a hypervisor executing on a server206 or a hypervisor executing on a client 240.

Some embodiments include a client device 240 that displays applicationoutput generated by an application remotely executing on a server 206 orother remotely located machine. In these embodiments, the client device240 may execute a virtual machine receiver program or application todisplay the output in an application window, a browser, or other outputwindow. In one example, the application is a desktop, while in otherexamples the application is an application that generates or presents adesktop. A desktop may include a graphical shell providing a userinterface for an instance of an operating system in which local and/orremote applications can be integrated. Applications, as used herein, areprograms that execute after an instance of an operating system (and,optionally, also the desktop) has been loaded.

The server 206, in some embodiments, uses a remote presentation protocolor other program to send data to a thin-client or remote-displayapplication executing on the client to present display output generatedby an application executing on the server 206. The thin-client orremote-display protocol can be any one of the following non-exhaustivelist of protocols: the Independent Computing Architecture (ICA) protocoldeveloped by Citrix Systems, Inc. of Ft. Lauderdale, Fla.; or the RemoteDesktop Protocol (RDP) manufactured by the Microsoft Corporation ofRedmond, Wash.

A remote computing environment may include more than one server 206a-206 n such that the servers 206 a-206 n are logically grouped togetherinto a server farm 206, for example, in a cloud computing environment.The server farm 206 may include servers 206 that are geographicallydispersed while logically grouped together, or servers 206 that arelocated proximate to each other while logically grouped together.Geographically dispersed servers 206 a-206 n within a server farm 206can, in some embodiments, communicate using a WAN (wide), MAN(metropolitan), or LAN (local), where different geographic regions canbe characterized as: different continents; different regions of acontinent; different countries; different states; different cities;different campuses; different rooms; or any combination of the precedinggeographical locations. In some embodiments the server farm 206 may beadministered as a single entity, while in other embodiments the serverfarm 206 can include multiple server farms.

In some embodiments, a server farm may include servers 206 that executea substantially similar type of operating system platform (e.g.,WINDOWS, UNIX, LINUX, iOS, ANDROID, SYMBIAN, etc.) In other embodiments,server farm 206 may include a first group of one or more servers thatexecute a first type of operating system platform, and a second group ofone or more servers that execute a second type of operating systemplatform.

Server 206 may be configured as any type of server, as needed, e.g., afile server, an application server, a web server, a proxy server, anappliance, a network appliance, a gateway, an application gateway, agateway server, a virtualization server, a deployment server, a SecureSockets Layer (SSL) VPN server, a firewall, a web server, an applicationserver or as a master application server, a server executing an activedirectory, or a server executing an application acceleration programthat provides firewall functionality, application functionality, or loadbalancing functionality. Other server types may also be used.

Some embodiments include a first server 206 a that receives requestsfrom a client machine 240, forwards the request to a second server 206 b(not shown), and responds to the request generated by the client machine240 with a response from the second server 206 b (not shown.) Firstserver 206 a may acquire an enumeration of applications available to theclient machine 240 as well as address information associated with anapplication server 206 hosting an application identified within theenumeration of applications. First server 206 a can then present aresponse to the client's request using a web interface, and communicatedirectly with the client 240 to provide the client 240 with access to anidentified application. One or more clients 240 and/or one or moreservers 206 may transmit data over network 230, e.g., network 101.

FIG. 3 shows a high-level architecture of an illustrative desktopvirtualization system. As shown, the desktop virtualization system maybe single-server or multi-server system, or cloud system, including atleast one virtualization server 301 configured to provide virtualdesktops and/or virtual applications to one or more client accessdevices 240. As used herein, a desktop refers to a graphical environmentor space in which one or more applications may be hosted and/orexecuted. A desktop may include a graphical shell providing a userinterface for an instance of an operating system in which local and/orremote applications can be integrated. Applications may include programsthat execute after an instance of an operating system (and, optionally,also the desktop) has been loaded. Each instance of the operating systemmay be physical (e.g., one operating system per device) or virtual(e.g., many instances of an OS running on a single device). Eachapplication may be executed on a local device, or executed on a remotelylocated device (e.g., remoted).

A computer device may be configured as a virtualization server in avirtualization environment, for example, a single-server, multi-server,or cloud computing environment. Virtualization server 301 illustrated inFIG. 3 can be deployed as and/or implemented by one or more embodimentsof the server 206 illustrated in FIG. 2 or by other known computingdevices. Included in virtualization server 301 is a hardware layer thatcan include one or more physical disks 304, one or more physical devices306, one or more physical processors 308, and one or more physicalmemories 316. In some embodiments, firmware 312 can be stored within amemory element in the physical memory 316 and can be executed by one ormore of the physical processors 308. Virtualization server 301 mayfurther include an operating system 314 that may be stored in a memoryelement in the physical memory 316 and executed by one or more of thephysical processors 308. Still further, a hypervisor 302 may be storedin a memory element in the physical memory 316 and can be executed byone or more of the physical processors 308.

Executing on one or more of the physical processors 308 may be one ormore virtual machines 332A-C (generally 332). Each virtual machine 332may have a virtual disk 326A-C and a virtual processor 328A-C. In someembodiments, a first virtual machine 332A may execute, using a virtualprocessor 328A, a control program 320 that includes a tools stack 324.Control program 320 may be referred to as a control virtual machine,Dom0, Domain 0, or other virtual machine used for system administrationand/or control. In some embodiments, one or more virtual machines 332B-Ccan execute, using a virtual processor 328B-C, a guest operating system330A-B.

Virtualization server 301 may include a hardware layer 310 with one ormore pieces of hardware that communicate with the virtualization server301. In some embodiments, the hardware layer 310 can include one or morephysical disks 304, one or more physical devices 306, one or morephysical processors 308, and one or more physical memory 316. Physicalcomponents 304, 306, 308, and 316 may include, for example, any of thecomponents described above. Physical devices 306 may include, forexample, a network interface card, a video card, a keyboard, a mouse, aninput device, a monitor, a display device, speakers, an optical drive, astorage device, a universal serial bus connection, a printer, a scanner,a network element (e.g., router, firewall, network address translator,load balancer, virtual private network (VPN) gateway, Dynamic HostConfiguration Protocol (DHCP) router, etc.), or any device connected toor communicating with virtualization server 301. Physical memory 316 inthe hardware layer 310 may include any type of memory. Physical memory316 may store data, and in some embodiments may store one or moreprograms, or set of executable instructions. FIG. 3 illustrates anembodiment where firmware 312 is stored within the physical memory 316of virtualization server 301. Programs or executable instructions storedin the physical memory 316 can be executed by the one or more processors308 of virtualization server 301.

Virtualization server 301 may also include a hypervisor 302. In someembodiments, hypervisor 302 may be a program executed by processors 308on virtualization server 301 to create and manage any number of virtualmachines 332. Hypervisor 302 may be referred to as a virtual machinemonitor, or platform virtualization software. In some embodiments,hypervisor 302 can be any combination of executable instructions andhardware that monitors virtual machines executing on a computingmachine. Hypervisor 302 may be Type 2 hypervisor, where the hypervisorexecutes within an operating system 314 executing on the virtualizationserver 301. Virtual machines may then execute at a level above thehypervisor 302. In some embodiments, the Type 2 hypervisor may executewithin the context of a user's operating system such that the Type 2hypervisor interacts with the user's operating system. In otherembodiments, one or more virtualization servers 301 in a virtualizationenvironment may instead include a Type 1 hypervisor (not shown). A Type1 hypervisor may execute on the virtualization server 301 by directlyaccessing the hardware and resources within the hardware layer 310. Thatis, while a Type 2 hypervisor 302 accesses system resources through ahost operating system 314, as shown, a Type 1 hypervisor may directlyaccess all system resources without the host operating system 314. AType 1 hypervisor may execute directly on one or more physicalprocessors 308 of virtualization server 301, and may include programdata stored in the physical memory 316.

Hypervisor 302, in some embodiments, can provide virtual resources tooperating systems 330 or control programs 320 executing on virtualmachines 332 in any manner that simulates the operating systems 330 orcontrol programs 320 having direct access to system resources. Systemresources can include, but are not limited to, physical devices 306,physical disks 304, physical processors 308, physical memory 316, andany other component included in hardware layer 310 of the virtualizationserver 301. Hypervisor 302 may be used to emulate virtual hardware,partition physical hardware, virtualize physical hardware, and/orexecute virtual machines that provide access to computing environments.In still other embodiments, hypervisor 302 may control processorscheduling and memory partitioning for a virtual machine 332 executingon virtualization server 301. Hypervisor 302 may include thosemanufactured by VMWare, Inc., of Palo Alto, Calif.; the XENPROJECThypervisor, an open source product whose development is overseen by theopen source XenProject.org community; HyperV, VirtualServer or virtualPC hypervisors provided by Microsoft, or others. In some embodiments,virtualization server 301 may execute a hypervisor 302 that creates avirtual machine platform on which guest operating systems may execute.In these embodiments, the virtualization server 301 may be referred toas a host server. An example of such a virtualization server is theXENSERVER provided by Citrix Systems, Inc., of Fort Lauderdale, Fla.

Hypervisor 302 may create one or more virtual machines 332B-C (generally332) in which guest operating systems 330 execute. In some embodiments,hypervisor 302 may load a virtual machine image to create a virtualmachine 332. In other embodiments, the hypervisor 302 may execute aguest operating system 330 within virtual machine 332. In still otherembodiments, virtual machine 332 may execute guest operating system 330.

In addition to creating virtual machines 332, hypervisor 302 may controlthe execution of at least one virtual machine 332. In other embodiments,hypervisor 302 may present at least one virtual machine 332 with anabstraction of at least one hardware resource provided by thevirtualization server 301 (e.g., any hardware resource available withinthe hardware layer 310). In other embodiments, hypervisor 302 maycontrol the manner in which virtual machines 332 access physicalprocessors 308 available in virtualization server 301. Controllingaccess to physical processors 308 may include determining whether avirtual machine 332 should have access to a processor 308, and howphysical processor capabilities are presented to the virtual machine332.

As shown in FIG. 3, virtualization server 301 may host or execute one ormore virtual machines 332. A virtual machine 332 is a set of executableinstructions that, when executed by a processor 308, may imitate theoperation of a physical computer such that the virtual machine 332 canexecute programs and processes much like a physical computing device.While FIG. 3 illustrates an embodiment where a virtualization server 301hosts three virtual machines 332, in other embodiments virtualizationserver 301 can host any number of virtual machines 332. Hypervisor 302,in some embodiments, may provide each virtual machine 332 with a uniquevirtual view of the physical hardware, memory, processor, and othersystem resources available to that virtual machine 332. In someembodiments, the unique virtual view can be based on one or more ofvirtual machine permissions, application of a policy engine to one ormore virtual machine identifiers, a user accessing a virtual machine,the applications executing on a virtual machine, networks accessed by avirtual machine, or any other desired criteria. For instance, hypervisor302 may create one or more unsecure virtual machines 332 and one or moresecure virtual machines 332. Unsecure virtual machines 332 may beprevented from accessing resources, hardware, memory locations, andprograms that secure virtual machines 332 may be permitted to access. Inother embodiments, hypervisor 302 may provide each virtual machine 332with a substantially similar virtual view of the physical hardware,memory, processor, and other system resources available to the virtualmachines 332.

Each virtual machine 332 may include a virtual disk 326A-C (generally326) and a virtual processor 328A-C (generally 328.) The virtual disk326, in some embodiments, is a virtualized view of one or more physicaldisks 304 of the virtualization server 301, or a portion of one or morephysical disks 304 of the virtualization server 301. The virtualizedview of the physical disks 304 can be generated, provided, and managedby the hypervisor 302. In some embodiments, hypervisor 302 provides eachvirtual machine 332 with a unique view of the physical disks 304. Thus,in these embodiments, the particular virtual disk 326 included in eachvirtual machine 332 can be unique when compared with the other virtualdisks 326.

A virtual processor 328 can be a virtualized view of one or morephysical processors 308 of the virtualization server 301. In someembodiments, the virtualized view of the physical processors 308 can begenerated, provided, and managed by hypervisor 302. In some embodiments,virtual processor 328 has substantially all of the same characteristicsof at least one physical processor 308. In other embodiments, virtualprocessor 308 provides a modified view of physical processors 308 suchthat at least some of the characteristics of the virtual processor 328are different than the characteristics of the corresponding physicalprocessor 308.

Intermediated Retrieval of Content

FIG. 4 shows a remote access environment in the network 101 comprising afirst computing device 401, a second computing device 402, and anintermediary computing device 403. The first computing device 401, thesecond computing device 402, and the intermediary computing device 403may be computing devices the same as or similar to the computing device201. For example, the first computing device 401 may be a laptop,whereas the second computing device 402 and the intermediary computingdevice 403 may be servers. The first computing device 401, the secondcomputing device 402, and the intermediary computing device 403 may beconnected over the network 101. Communications over the network 101 maybe via one or more protocols, such as HTTP (e.g., HTTP3, also referredto as “QUIC”), Transmission Control Protocol (TCP), User DatagramProtocol (UDP), or the like, including a combination thereof. Thenetwork 101 may comprise a variety of switches, routers, other computingdevices, and the like, though these elements are not shown forsimplicity.

The second computing device 402 may store content 404, which may be,e.g., a web page, application, or the like. Computing devices, such asthe first computing device 401, may be configured to request the content404 from the second computing device 402. The second computing device402 may be configured to send the content 404 to one or more computingdevices, such as the first computing device 401. Such communications maybe via an application, such as a server operating system, a web browser,or the like. For example, the first computing device 401 may use a webbrowser to access the content 404, hosted by a server applicationexecuting on the second computing device 402, over the network 101.

The first computing device 401 may access the content 404 on the secondcomputing device 402 via the intermediary computing device 403. Anintermediary computing device, such as the intermediary computing device403, may be a computing device configured to provide remote access toapplications executing on the intermediary computing device, such asoffice suite applications, web browsers, and the like. The intermediarycomputing device 403 may comprise or use a virtualization server, suchas the virtualization server 301. Such remote access may be via a remoteaccess protocol, such as the ICA protocol. Applications provided oversuch a remote access protocol may be used to access content on othercomputing devices, such as the content 404 on the second computingdevice 402.

Each hop between two or more computing devices, such as the firstcomputing device 401 and the second computing device 402, may beassociated with network state metrics, such as a latency, a round triptime (RTT), a bandwidth, packet loss, web page loading time, and thelike. As used herein, a hop may refer to a portion of a connectionbetween a sending and receiving computing device such that, for example,transmissions between the first computing device 401 and theintermediary computing device 403 over a first protocol may comprise afirst hop, whereas transmissions between the intermediary computingdevice 403 and the second computing device 402 may comprise a secondhop. Such network state metrics may be associated with transmissiondelays (e.g., time required to transmit signal via wire), processingdelays (e.g., delays associated with network interface cards in acomputing device), routing delays (e.g., transmission of packets betweendifferent networks), physical limitations (e.g., a particular wire onlybeing capable of transmitting so many packets per second), and the like.For example, a direct connection 405 a between the first computingdevice 401 and the second computing device 402 may be associated with300 milliseconds of latency. A first intermediate connection 405 bbetween the first computing device and the intermediary computing device403 may be associated with 300 milliseconds of latency. A secondintermediate connection 405 c between the intermediary computing device403 and the second computing device 402 may be associated with 50milliseconds of latency. The first intermediate connection 405 b may bevia a remote access protocol (e.g. the ICA protocol), whereas the secondintermediate connection 405 c may be via HTTP3. While the latency viathe direct connection 405 a may be less than the latency associated withthe first intermediate connection 405 b and the second intermediateconnection 405 c, a user may have a better experience with the firstintermediate connection 405 b and the second intermediate connection 405c as compared to the direct connection 405 a. For example, because thevideo transmitted by the remote access protocol over the firstintermediate connection 405 b may be configured to mitigate theundesirable effects of latency, a user may perceive the connection viathe first intermediate connection 405 b and the second intermediateconnection 405 c as snappier as compared to the direct connection 405 a.As another example, the content 404 may be a web page with a largequantity of elements, and the second intermediate connection 405 cbetween the intermediary computing device 403 and the second computingdevice 402 may, because it is associated with a lower latency, make allof the various elements of the web page appear to load faster despitethe fact that the first intermediate connection 405 b and the secondintermediate connection 405 c have a total latency greater than thedirect connection 405 a.

FIG. 5 shows three paths between the first computing device 401 and thesecond computing device 402. Though only four computing devices areshown in FIG. 5, an infinite number of computing devices (e.g.,switches, routers, or the like) may be involved in communicationsbetween the first computing device 401 and the second computing device402. For example, a first path 501 is depicted in FIG. 5 as a directline from the first computing device 401 to the second computing device402; however, such a path may involve numerous switches, routers, andthe like, including changes in protocol.

The first path 501 may comprise a single hop, a HTTP3 connection 504 a,associated with a first latency 506 a of 300 ms. The first path 501 maybe referred to as a direct path because it does not comprise remoteaccess to content via an intermediary computing device.

A second path 502 may comprise two hops. A first hop of the second path502, a remote access connection 504 b, may connect the first computingdevice 401 to the intermediary computing device 403. A second hop of thesecond path 502, an HTTP connection 504 c, may connect the intermediarycomputing device 403 to the second computing device 402. The remoteaccess connection 504 b may be associated with a second latency 506 b of300 milliseconds, and the HTTP connection 504 c may be associated with athird latency 506 c of 50 milliseconds.

A third path 503 may comprise three hops. A first hop of the third path503, a remote access connection 504 d, may connect the first computingdevice 401 to a second intermediary computing device 505. A second hopof the third path 503, a proprietary connection 504 e, may connect thesecond intermediary computing device 505 to the intermediary computingdevice 403. A third hop of the third path 503, an HTTP connection 504 f,may connect the intermediary computing device 403 to the secondcomputing device 402. The remote access connection 504 d may beassociated with a fourth latency 506 d of 150 milliseconds, theproprietary connection 504 e may be associated with a fifth latency 506e of 200 milliseconds, and the HTTP connection 504 f may be associatedwith the third latency 506 c of 50 milliseconds.

User experience may be improved through use of remote accessconnections. For example, a user of the first computing device 401 mayload content on the second computing device 402 and have a better userexperience by remotely accessing a web browser on the intermediarycomputing device 403 via the remote access connection 504 b and loadingthe content in the remote browser via the HTTP connection 504 c, ratherthan directly retrieving the content via the HTTP3 connection 504 a. Inother words, as discussed above, a user may have a better userexperience loading content via an intermediary computing device (e.g.,the intermediary computing device 403) as compared to directlyretrieving such content. For example, packets transmitted via the firstpath 501 may result in a relatively poor Quality of Service (QOS)treatment across different networks, whereas packets transmitted via thethird path 503 may receive a relatively better quality QOS treatment. Asanother example, the remote access connection 504 b may be configured tomitigate the effects of the second latency 506 b, such that the user mayperceive loading of the content from the second computing device 402 tobe faster despite the fact that the sum of the second latency 506 b andthe third latency 506 c is greater than the first latency 506 a.

The latency values shown in FIG. 5 may be influenced by a variety offactors, including the location of the computing devices, the processingand transmission speed of the computing devices, the physical propertiesof connections between the computing devices, and the like. For example,the latency improvements of the second path 502, as compared to thefirst path 501, may in part be realized because the remote accessconnection 504 b may be treated with a higher QOS on some networks,because routers/switches in the second path 502 are faster and/or morepowerful than those on the first path 501, because more bandwidth may beallocated to computing devices along the second path 502 than the firstpath 501, or the like.

Path Selection

FIG. 6 is a flow chart which may be performed with respect to the firstcomputing device 401 and the second computing device 402. All or some ofthe steps shown in FIG. 6 may be performed by the first computing device401, the second computing device 402, the intermediary computing device403, and/or an administrative computing device. For example, all or someof the steps shown in FIG. 6 may be performed by the intermediarycomputing device 403 such that it may cause the first computing device401 to connect through it to the second computing device 402.

In step 601, a request for content (e.g., the content 404 as stored bythe second computing device 402) from a requesting computing device maybe received or otherwise communicated. Additionally and/oralternatively, step 601 may comprise predicting that such a request forcontent may be likely (e.g., determining that a user of the computingdevice 401 has performed authentication steps required to connect withthe second computing device 402) and/or otherwise determining theexistence of a request for content. For example, the request for contentmay be a user of a computing device clicking on an icon and/or similarlink associated with the content. The request for content may originatefrom the computing device hosting the content (e.g., the secondcomputing device 402), such that the request for content may comprise anindication that, e.g., the second computing device 402 wishes totransmit the content to the first computing device 401. The request maybe received via one or more intermediary computing devices such that,for example, the request for content may be received via an applicationexecuting in a remote access environment (e.g., as shown in FIG. 4). Therequest may be initiated by, for example, determining that a user ishovering over an icon, link, or similar user interface elementassociated with the content. The request may be initiated by a userlogging into a service, such as a remote access service. The request maybe based on manual indications by a user of desired content: forexample, a user might indicate, using an application, that the userregularly uses certain remote applications.

In step 602, network state metrics may be determined. Metrics, such asthe network state metrics, may provide information regarding one or morepaths (including portions of such paths), and such metrics may be usedto select a path amongst a plurality of available paths. For example,positive network state metrics (e.g., high bandwidth, low latency) for apath may encourage use of that path to retrieve content. Network statemetrics may correspond to one or more paths (e.g., the first path 501,the second path 502, and/or the third path 503) in a network (e.g., thenetwork 101). Network state metrics may be determined for a plurality ofpaths, including hops in one or more paths. For example, network statemetrics may comprise bandwidth, RTT, and/or latency information for eachhop in one or more paths between the first computing device 401 and theintermediary computing device 403. As another example, network statemetrics may be collected for every hop in a network, a subset of hops ina network (e.g., those predicted to be used to connect the firstcomputing device 401 and the second computing device 402), or the like.Network state metrics may be determined for a variety of protocols, suchas HTTP, HTTP3, and the like, such that, e.g., a latency may bedetermined for HTTP3 and HTTP over the same hop in a path. Network statemetrics may be based on information associated with requesting computingdevice, such as a location of the requesting computing device, theInternet Protocol (IP) address of the requesting computing device, orthe like.

In step 603, remote access protocol metrics may be determined. Remoteaccess protocol metrics may comprise metrics the same or similar to thenetwork state metrics but with respect to remote access to applicationsover a remote access protocol. For example, packets via a first protocol(e.g., HTTP3) may be associated with a latency of 200 milliseconds,whereas packets associated with a remote access protocol (e.g., ICA) maybe associated with a latency of 100 milliseconds. This difference inlatency may relate to different processing/transmission requirementsassociated with each protocol. As such, the remote access protocolmetrics may correspond to latency, bandwidth, or similar measurementsmade between links using a remote access protocol. The remote accessprotocol metrics may also comprise other measurements associated withremote access to an application, such as frames per second videotransmitted to a computing device using the remote access protocol toaccess an application, the CPU load and/or available memory of thecomputing device executing the remotely-accessed application, and thelike. Such remote access protocol metrics may be determined by, e.g., ahypervisor, such as the hypervisor 302.

In step 604, user experience metrics may be determined for one or morepaths. User experience metrics may be a quantification of a user'sexperience with content (e.g., the content 404) as provided over anetwork (e.g., the network 101). User experience metrics may comprise aweighted form of the network state metrics and the remote accessprotocol metrics associated with one or more paths, as weighted by therelative importance of such metrics for user experience. For example,because a user might more readily notice lag in a remote accessenvironment as compared to lag when retrieving elements of a web page,remote access protocol metrics may be weighted as more important thannetwork state metrics, such that low latency may be more important forpaths delivering remote access protocol information (e.g., video and/oraudio corresponding to a remote application) as opposed to pathsdelivering content (e.g., paths delivering HTTP content to the remoteapplication). As will be described below, particularly with respect toFIG. 7 and FIG. 9, user experience metrics may be determined based onhistorical user behavior such that historical user experiences withvarious paths may influence how user experience metrics are determined.

One of the user experience metrics for a path may comprise a weightedsum of the network state metrics and/or the remote access protocolmetrics associated with the path. For example, where the network statemetrics and the remote access protocol metrics comprise latencymeasurements, a user experience metric may be found using the followingformula, where u_(p) is a user experience metric for a path, w is aunique weighting value, r is a remote access protocol latency, and n isa network state latency:

$u_{p} = {{\sum\limits_{i = 0}^{I}{w_{i}r_{i}}} + {\sum\limits_{j = 0}^{J}{w_{j}n_{j}}}}$

Weighting values, such as w in the above equation, may be determinedbased on the relative impact of a given network state metric and/orremote access protocol metric on user experience. For example, arelatively high remote access protocol latency may be undesirablebecause a user may more readily notice the latency (e.g., by detecting adelay in mouse movement using the first computing device 401 and acorresponding mouse movement in a video stream received from theintermediary computing device 403). As another example, and in contrast,a relatively high network state metric may be more permissible where thelatency would correspond in delayed retrieval of content that need notbe loaded immediately (e.g., content which is not displayed until theuser has viewed other content for ten seconds).

Weighting values such as w may be dynamically adjusted, e.g., by amachine learning algorithm, based on a history of user experience and/oruser behavior. If certain network state metrics and/or remote accessprotocol metrics are associated with poor user experience and/orundesirable user behavior (e.g., an increased likelihood that a userstops accessing the content, attempts to re-connect to the content, idlymoves their mouse around without engaging with the content, etc.), suchnetwork state metrics and/or remote access protocol metrics may beweighted highly. As such, the user experience metrics may, over time,more accurately reflect the importance of particular paths and/ormetrics with respect to user experience.

User experience metrics may be reproduced as value corresponding towhether a particular path, or portion of a path, has a satisfactory userexperience. For example, a sigmoid transformation may be applied to theequation for u_(p) provided above, such that the function ranges from 0to 1, with 1 being satisfactory, 0 being unsatisfactory, and v being avariable which may be modified to define the range of values of u may besatisfactory or unsatisfactory:

$y = {{\frac{1}{1 + e^{- z}}\mspace{14mu} z} = {{u_{p} + v} = {{\sum\limits_{i = 0}^{I}{w_{i}r_{i}}} + {\sum\limits_{j = 0}^{J}{w_{j}n_{j}}} + v}}}$

One or more thresholds may be established which define a good userexperience versus a bad user experience. The value y in the aboveequation may output values in the range from zero to one, such that yapproaches 1 as z approaches negative infinity, y approaches ½ as zapproaches zero, and y approaches 0 as z approaches infinity. Values maybe set such that y<0.5 may indicate an unacceptable user experience, andy>=0.5 may indicate an acceptable user experience. As another example,the constant v in the equation for y above may be modified such thaty=0.5 is the minimum acceptable user experience, such that values belowor equal to y=0.5 indicate an unacceptable user experience, and valuesabove y=0.5 indicate an acceptable user experience. As another example,a predetermined value of u in the above equation may be determined as aminimum acceptable latency, such that any latency greater than thatminimum acceptable latency may indicate an unacceptable user experience.The function above may be referred to as a sigmoid function and/or alogistic function.

In step 605, based on the network state metrics, the remote accessprotocol metrics, and/or the user experience metrics, a path may beselected. The path may be selected from a plurality of paths between,e.g., the first computing device 401 and the second computing device402. For example, based on the network state metrics, the remote accessprotocol metric, and/or the user experience metrics, a network statecalculation may be determined which comprises an estimation of userexperience for each of a plurality of paths. The path selected need notbe the path with the lowest latency and/or highest bandwidth, and maycomprise the path associated with a highest user experiencedetermination. For example, a relatively slower path may be selected ifportions of the path more important for user experience (e.g., pathsbetween the first computing device 401 and the intermediary computingdevice 403) are nonetheless faster as compared to other paths. Asanother example, step 605 may comprise making a user experiencedetermination for a plurality of paths and selecting, from the paths, apath with the best user experience determination.

The selected path may be for the same or a different session, a group ofusers, a group of computing devices, or the like. A user may accesscontent in one or more sessions, which may be discrete periods of timecorresponding to different instances in which the user accesses content.Thus, a selected path for a first session (e.g., access to given contenton Monday) may be different than a selected path for a second session(e.g., access to the same content on Tuesday). For example, the firstcomputing device 401 may already be receiving first content from thesecond computing device 402, and the selected path may be for receipt ofsecond content. As another example, the first computing device 401 mayreceive content from the second computing device 402 via a first path,and the selected path may be stored in memory for use when the firstcomputing device 401 requests second content, in a different session,from the second computing device 402. The selected path may be used forall users and/or all computing devices in a predetermined group. Forexample, the path may be selected and stored for use for all computingdevices located in China that request content from computing deviceslocated in Canada. As another example, the path may be selected for allusers associated with a particular tier of a subscription service. Asanother example, the path may be selected for all users in an office.

In step 606, the requesting computing device may be connected, via thepath selected in step 605, to the content. Causing the connection maycomprise transmitting, to the computing device, instructions to connectto an intermediary computing device, and/or may comprise instructions toan intermediary computing device to provide, to a computing device,remote access to an application. For example, the first computing device401 may be connected, via the path, to the second computing device 402by sending, to the first computing device 401, an instruction to connectto a particular server comprising a first hop in the selected path. Asanother example, an ICA file may be created such that, when the file isopened by the first computing device 401, the first computing device 401establishes a connection, via a remote access protocol, with anintermediate computing device along the selected path.

Connecting a requesting computing device, via the path selected in 605,to the content may comprise presenting, and/or modifying, one or moreuser interface elements on the requesting computing device. For example,the first computing device 401 may show a plurality of icons, onecorresponding to the content 404, and such an icon may be modified suchthat interaction (e.g., clicking, hovering over) with the icon causesaccess to the content 404 via the path selected in step 605. In thismanner, the user of the requesting computing device need not know thatthe path is being selected. This may be advantageous to make access tothe content appear seamless, and/or in environments where the userdesiring the content is not particularly tech-savvy.

In step 607, user input data associated with access to the content maybe received. Such user input data may comprise information regarding auser moving a mouse, typing on a keyboard, and/or otherwise engagingwith the requested content. Such user input data may be received via anintermediary computing device, such as the intermediary computing device403. For example, software executing on the intermediary computingdevice 403 enabling the intermediary computing device 403 to provide, tothe first computing device 401, remote access to an application may alsoreceive input data from the first computing device 401. Such user inputdata may be in any format, and may be summary data (e.g., clicks perminute, an indication of a time period when a user's mouse has notmoved, or the like).

In step 608, a user behavior analysis may be performed. A user'sbehavior with respect to content may be reflective of whether theiraccess to content is unsatisfactory. For example, if content is loadingtoo slowly, a user may move the mouse around in absentminded patterns,may access a window different than a window associated with the content,may not provide any keyboard input for a predetermined period of time,or the like. Thus, a user behavior analysis may provide an indicationthat a new path selection may be desirable to improve the user'sexperience with the content. The user behavior analysis may compriseanalyzing available data, such as user input data, to determine how auser is interacting with requested content. After a user has interactedwith a first portion of content at a first time, a user may not interactwith the same or a different portion of content at a second timebecause, e.g., the content is loading too slowly and/or is difficult tointeract with. For example, the user input data may indicate that a userthat formerly was interacting with the content is now idly moving theirmouse without clicking or using a mouse wheel, that a long time haselapsed since the user typed anything into their keyboard or clicked amouse, and/or where the user attempts to re-open and/or close a sessionwith an intermediate computing device. Such user behavior may indicate,e.g., that the user experience with the content is poor, and thatanother path to retrieve the content may be necessary to improve theuser's experience. Additionally and/or alternatively, the user behavioranalysis may be performed based on network state metrics and/or remoteaccess protocol metrics. For example, if the network state metricsindicate that data has been downloaded for a predetermined period oftime (e.g., that a user has been trying to download the same content viathe same path for longer than five minutes), and if the user input datasuggests that the user is still active, the user may not be consumingcontent and therefore have a poor user experience.

Because user behavior may be the result of a large number of factorsunrelated to the retrieval of content, the user behavior analysis mayconsider multiple factors, and may be configured to learn over timewhich of the multiple factors are indicative of issues with theretrieval of content via a given path. For example, based on learningthat a user has a habit of idly moving a mouse while accessing websites,the user behavior analysis may be configured to ignore and/or discountsuch mouse movement. As another example, content by the user may haveregions associated with mouse clicks (e.g., links on a web page), suchthat mouse clicks in such regions may be associated with content access,whereas mouse clicks outside of such regions may be indicative of issueswith content access. The user behavior analysis may be additionallyand/or alternatively based on the content retrieved. For example, it maybe learned over time that user input for an interactive website (e.g., abrowser-based game) is likely to be common, such that an absence of userinput data may indicate content access issues, whereas user input datafor a static website (e.g., a news article) is likely to be relativelyless common.

In step 609, it is determined whether the user behavior analysisperformed in step 608 suggests a poor user experience. If not, thecontent is provided in step 611, and the flow chart ends. If so, theflow chart proceeds to step 610. In step 610, one or more causes for theuser behavior analyzed in step 608 (e.g., the poor user experience) maybe determined. If a poor connection is suggested by the user behavior instep 608, then the system determines which portions of the path used todeliver the content may be at fault. For example, a particular hop in apath (e.g., a hop between the intermediary computing device 403 and thesecond computing device 402) may be undesirably slow, causing the userto be bored and move their mouse around erratically. Such causes may belater used to determine a different path for retrieval of the requestedcontent. For example, such causes may be used to train a machinelearning algorithm used in step 604 to determine the user experiencemetrics, such that future user experience metrics are based onhistorical user experience with various paths.

After the one or more causes for the poor connection are determined instep 610, path re-selection may be performed by returning to step 602.As such, based on determining that a user's experience with retrievingcontent via a certain path is poor, the path to retrieve such contentmay be re-selected.

As an example of the process depicted in FIG. 7, a first computingdevice may receive, from a second computing device (e.g., a mobiledevice, such as a smartphone) a request for content stored on the firstcomputing device. For example, the second computing device may request aweb page stored on the first computing device. A first state of a firstconnection between the first computing device and the second computingdevice may be determined. A second state of a second connection betweenthe first computing device and the second computing device may bedetermined. The first connection may be configured to transfer thecontent from the first computing device to the second computing devicevia a first protocol (e.g., directly, and via HTTP). The secondconnection may be different from the first connection, may comprise atleast one intermediary computing device, and may be configured totransfer the content from the first computing device to the secondcomputing device via a second protocol (e.g., using a remote accessprotocol). The at least one intermediary computing device may beconfigured to deliver video (e.g., via the second protocol)corresponding to an application comprising the content (e.g., videocomprising a web browser displaying a requested web page). Determiningsuch states may comprise determining a plurality of latency valuesassociated with devices in the connection, such as the first computingdevice, second computing device, and/or the at least one intermediarycomputing device, and determining a projected latency for the connectionbased on the plurality of latency values (e.g., by weighting theplurality of latency values based on a history of user experiencemetrics). Based on a comparison of the first state and the second state,one of the first connection and the second connection may be selected.The connection selected need not be the connection with the leastlatency (e.g., the least projected latency), and may be the connectionassociated with a superior user experience metric. For example, a userexperience metric may be determined, and a connection may be selectedbased on that user experience metric, even though the connection isassociated with a higher latency than another connection. The selectedconnection may be used to provide the content from the first computingdevice to the second computing device, which may thereby avoid delaysassociated with the non-selected connection. Based on user input datareceived from the second computing device, an indication of userbehavior may be determined, and, if the user behavior indicatesunsatisfactory network conditions, a third connection may be selected,and/or the non-selected connection may be selected.

FIG. 7 shows a process flow chart with respect to the first computingdevice 401 and the second computing device 402. FIG. 7 is divided intofour stages: a data input stage 701, a connection state model stage 702,a network state calculations stage 703, and an output stage 704. All orsome of the processes and/or steps shown in FIG. 7 may be performed bythe first computing device 401, the second computing device 402, theintermediary computing device 403, and/or an administrative computingdevice. The process flow chart shown in FIG. 7 may generally correspondto the steps shown in FIG. 6. For example, the process described withrespect to the data input stage 701 and the connection state model stage702 may be the same or similar to that described with respect to, e.g.,step 602, step 603, and step 604. As another example, the processdescribed with respect to the network state calculations stage 703 andthe output stage 704 may be the same or similar to that described withrespect to, e.g., step 605 and step 606.

In the data input stage 701, the network state metrics and the remoteaccess protocol metrics 705 may be weighted, in step 706, to become userexperience metrics 707. Such metrics may then proceed to the connectionstate model stage 702.

In the connection state model stage 702, the network state metrics andthe remote access protocol metrics 705, and/or the user experiencemetrics 707 may be evaluated by a connection state model 708. Theconnection state model 708 may be used to train (e.g., weight and/orotherwise modify) network state metrics, remote access protocol metrics,and/or user experience metrics based on user behavior analysis results715, and may thereby emphasize which of such metrics matter with respectto user experience. As such, the connection state model 708 update suchmetrics in step 709. For example, based on determining that a particularnetwork state metric (e.g., the bandwidth of a particular hop) is notparticularly important with respect to user experience, the connectionstate model 708 may weight that particular network state metric in amanner which discounts it relative to other network state metrics. Assuch, the connection state model stage 702 may be used to modify datareceived in the data input stage 701 based on, e.g., a history of userexperience given particular network conditions. As another example,based on a history of user experiences, the weights of the userexperience metrics 707 may be updated to prioritize the latency ofremote access protocols over other protocols. After updating suchmetrics in step 709, the network state metrics and the remote accessprotocol metrics 705, the user experience metrics 707, and/or any otherdata may proceed to the network state calculations stage 703.

In the network state calculations stage 703, a network state calculation710 may use the metrics from the connection state model stage 702 todetermine projected user experiences across one or more paths availablebetween, e.g., the first computing device 401 and the second computingdevice 402. Such projected user experiences are indicated as userexperience determinations for one or more paths 711. The network statecalculation 710 may comprise using the metrics received from theconnection state model stage 702 to rank a plurality of available pathsbased on their likely user experience. For example, there may be threepaths (e.g., the paths depicted in FIG. 5) available to a user of afirst computing device, and the network state calculation 710 maycomprise determining, using the metrics from the data input stage 701 asweighted in step 706 and/or as modified by the connection state model708, a ranking, based on the metrics, of each path. Such a ranking maybe, for example, a projected user experience (e.g., from best to worst)for each path of a plurality of paths available. As such, the resultinganalysis, the user experience determination for the one or more paths711, may comprise, e.g., a ranking of paths based on their projecteduser experience. The user experience for the one or more paths 711 maythen proceed to the output stage 704.

In the output stage 704, path selection 712 may be performed. Forexample, if the user experience determination for the one or more paths711 is a ranking of a plurality of paths based on their likely userexperience, then the highest-ranked path may be selected. The processmay then proceed back to the data input stage 701, and more particularlyretrieval of user input data 713.

Returning to the data input stage 701, the user input data 713 may beretrieved, and analysis 714 of the user input data 713 may be performed,such that, in the connection state model stage 702, user behavioranalysis results 715 may be determined. Such a process may be the sameor similar to the process described with respect to, e.g., step 607,step 608, step 609, and step 610. The user behavior analysis results 715may be used by the connection state model 708 to train 716 the metricssent to the network state calculation 710. For example, if the pathselection 712 results in the user behavior analysis results 715indicating a poor user experience, then a similar path selection may bediscouraged by appropriately modifying weighting values applied by theconnection state model 708 during the updating in step 709.

The process flow diagram shown in FIG. 7 illustrates how the system maylearn, over time, how to select paths with better user experience. Amachine learning algorithm may be implemented to perform such steps. Forexample, a machine learning algorithm may be trained based on the userbehavior analysis results 715, and may further be implemented as part ofthe analysis 714 and/or the updating in step 709 such that metrics(e.g., the network state metrics and the remote access protocol metrics705, and/or the user experience metrics 707) are modified based on theuser behavior analysis results 715. The machine learning algorithm maythereby learn when implementation of remote access to intermediarydevices may improve a user's experience of retrieving content.

FIG. 8 shows a high level flow chart for path selection. FIG. 8 may be asimplified version of FIG. 6 such that the steps of FIG. 6 may be thesame or similar as depicted in FIG. 8, and is simplified in part toillustrate how user feedback may be used to learn, over time, userexperience. Network state metrics 801 and remote access protocol metrics802 may be retrieved in step 602 and step 603 and as described withrespect to the data input stage 701, may be provided to a userexperience calculator 803. The user experience calculator 803 may be,e.g., software executing on the first computing device 401, the secondcomputing device 402, and/or the intermediary computing device 403. Forexample, the user experience calculator 803 may be part of remote accesssoftware executing on the first computing device 401 which enablesremote access to applications (e.g., web browsers) hosted on theintermediary computing device 403. The user experience calculator 803,which may perform functions described with respect to the connectionstate model stage 702, the network state calculations stage 703, and/orstep 604, may determine one or more user experience metrics, perform oneor more network state calculations, and/or may rank available pathsbased on such user experience metrics and/or network state calculations.Output from the user experience calculator 803 may be sent to a pathselector 804 which, similar to the process described in step 605 and/orin the output stage 704, may select a path of one or more paths. Theselected path from the path selector 804 may be implemented by the pathimplementation step 805, which may perform a process the same or similaras step 605 and/or as described with respect to the output stage 704.Then, user feedback 806, as described with respects to step 607, step608, step 609, step 610, the data input stage 701, and/or the connectionstate model stage 702 may be collected and used by the user experiencecalculator 803.

FIG. 9 shows user behavior analysis based on the user input data 713.The user input data 713 may be input into a user behavior analysisalgorithm, which may be configured to determine one or more indicia ofuser experience. Five examples of indications of poor user experienceare provided. If the user is active but there is relatively low downlinkdata being transmitted (box 902 a), this may indicate that a user isusing a remote access application but not downloading and/or otherwiseengaging with content (e.g., the content 404). If the user is moving amouse around aimlessly (e.g., moving at a velocity and/or accelerationwith a variance over time that exceeds a threshold, moving aroundportion of a display not associated with content, moving in definedpatterns, or the like) (box 902 b), and/or if there is a long delaybetween when the user clicks using their mouse (box 902 c), such actionsmay indicate that the user is not engaging with requested content, andthat the user's experience with the content may be poor. If the userattempts to re-open a remote access session and/or close an existingremote access session (box 902 d), such a step may indicate that theuser's experience in retrieving the content may be poor. If the user ispresented (e.g., via a user interface) with an opportunity to providefeedback regarding the user's retrieval of content (e.g., a thumbs upand/or thumbs down icon along with a request for feedback on networkconnectivity), and if the user responds negatively, such an indicationmay indicate that the user's experience in retrieving the content may bepoor. All such considerations may result in the user behavior analysisresults 715. The user behavior analysis results 715 may be stored in adatabase 903 which, e.g., may be used by the connection state model 708and/or during the network state calculation 710.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are described asexample implementations of the following claims.

What is claimed is:
 1. A method comprising: receiving, from a firstcomputing device, a request for content stored on a second computingdevice; determining a first state of a first connection between thefirst computing device and the second computing device, wherein thefirst connection is configured to transfer the content from the secondcomputing device to the first computing device via a first protocol;determining a second state of a second connection between the firstcomputing device and the second computing device, wherein the secondconnection is different from the first connection, and wherein thesecond connection comprises at least one intermediary device configuredto transfer the content from the second computing device to the firstcomputing device via a remote access protocol, different from the firstprotocol, that provides a remote access environment capable ofdisplaying the content; selecting, based on a comparison of the firststate and the second state, one of the first connection and secondconnection via which to transfer the content from the second computingdevice to the first computing device; and providing the content from thesecond computing device to the first computing device via the selectedconnection so as to improve transmission of the content from the secondcomputing device to the first computing device.
 2. The method of claim1, further comprising: receiving, from the first computing device, userinput data; and determining, based on the user input data, an indicationof user behavior associated with the content.
 3. The method of claim 2,further comprising: causing, based on determining that the user behaviorindicates unsatisfactory network conditions, the first computing deviceto retrieve, via a third connection, the content.
 4. The method of claim2, further comprising: providing, based on determining that the userbehavior indicates unsatisfactory network conditions, the content viathe first connection.
 5. The method of claim 1, wherein determining thesecond state comprises: determining a plurality of latency valuesassociated with a plurality of communications paths between the firstcomputing device, the at least one intermediary computing device, andthe second computing device; and determining, based on the plurality oflatency values, a projected latency of the second connection.
 6. Themethod of claim 5, wherein the plurality of latency values are weightedbased on a history of user experience metrics.
 7. The method of claim 1,wherein providing the content comprises: providing, to the firstcomputing device and via the at least one intermediary computing device,images corresponding to an application, executing in the remote accessenvironment, comprising the content.
 8. The method of claim 7, whereinthe images are delivered via the remote access protocol.
 9. The methodof claim 1, wherein comparing the first state and the second statecomprises: determining that a second user experience metric associatedwith the second state is greater than a first user experience metricassociated with the first state.
 10. A method comprising: determining afirst plurality of latencies associated with communications over anetwork, wherein the first plurality of latencies are associated with afirst protocol; determining a second plurality of latencies associatedwith communications over the network, wherein the second plurality oflatencies are associated with one or more remote access protocolsdifferent from the first protocol, wherein the one or more remote accessprotocols provide a remote access environment capable of displayingcontent stored by a second computing device; receiving, from a firstcomputing device, a request for the content; determining a plurality ofcommunications paths over the network between the first computing deviceand the second computing device; selecting, based on the first pluralityof latencies and the second plurality of latencies, a firstcommunications path within the network to provide the content of to thefirst computing device; causing the first computing device to establish,using the first communications path, a connection to retrieve thecontent of the second computing device; and providing the content fromthe second computing device to the first computing device via theconnection so as to improve transmission of the content from the secondcomputing device to the first computing device.
 11. The method of claim10, wherein selecting the first communications path comprises:determining, based on the first plurality of latencies and the secondplurality of latencies, a plurality of user experience metrics, whereinthe first communications path is selected based on the plurality of userexperience metrics.
 12. The method of claim 10, further comprising:receiving, from the first computing device, user input data; anddetermining, based on the user input data, an indication of userbehavior associated with the content.
 13. The method of claim 12,further comprising: causing, based on determining that the user behaviorindicates unsatisfactory network conditions, the first computing deviceto retrieve the content via a second communications path, wherein thesecond communications path comprises an intermediary computing deviceconfigured to transmit video, corresponding to the content, to the firstcomputing device.
 14. The method of claim 12, further comprising:causing, based on determining that the user behavior indicatesunsatisfactory network conditions, the first computing device toretrieve, via the first protocol, the content.
 15. A computing devicecomprising: one or more processors; and memory storing instructionsthat, when executed by the one or more processors, cause the computingdevice to: receive, from a mobile computing device, a request forcontent stored on the computing device; determine a first state of afirst connection between the computing device and the mobile computingdevice, wherein the first connection is configured to transfer thecontent from the computing device to the mobile computing device via afirst protocol; determine a second state of a second connection betweenthe mobile computing device and the computing device, wherein the secondconnection is different from the first connection, and wherein thesecond connection comprises at least one intermediary device configuredto transfer the content from the computing device to the mobilecomputing device via a remote access protocol, different from the firstprotocol, that provides a remote access environment capable ofdisplaying the content; select, based on a comparison of the first stateand the second state, one of the first connection and second connectionvia which to transfer the content from the computing device to themobile computing device; and provide the content from the computingdevice to the mobile computing device via the selected connection so asto improve transmission of the content from the computing device to themobile computing device.
 16. The computing device of claim 15, whereinthe instructions, when executed by the one or more processors, furthercause the computing device to: receive, from the mobile computingdevice, user input data; and determine, based on the user input data, anindication of user behavior associated with the content.
 17. Thecomputing device of claim 16, wherein the instructions, when executed bythe one or more processors, further cause the computing device to:cause, based on determining that the user behavior indicatesunsatisfactory network conditions, the mobile computing device toretrieve, via a third connection, the content.
 18. The computing deviceof claim 16, wherein the instructions, when executed by the one or moreprocessors, further cause the computing device to: provide, based ondetermining that the user behavior indicates unsatisfactory networkconditions, the content via the non-selected connection.
 19. Thecomputing device of claim 16, wherein the instructions, when executed bythe one or more processors, further cause the computing device to:determine a plurality of latency values associated with a plurality ofcommunications paths between the mobile computing device, the at leastone intermediary computing device, and the computing device; anddetermine, based on the plurality of latency values, a projected latencyof the second connection, wherein determining the second state is basedon the projected latency of the second connection.
 20. The computingdevice of claim 19, wherein the plurality of latency values are weightedbased on a history of user experience metrics.